A Comparative Study of a Practical Stochastic Clustering Method with Traditional Methods.

On Detecting Clustered Anomalies Using SCiForest.

Mass estimation and its applications.

Learning Naive Bayes Classifiers for Music Classification and Retrieval.

Mass estimation and its applications.

On Feature Combination for Music Classification.

Best papers from the 12th Pacific%2DAsia conference on knowledge discovery and data mining %28PAKDD2008%29.

FaSS Ensembles for Stable Learners.

Boosting Support Vector Machines Successfully.

Advances in Knowledge Discovery and Data Mining

Isolation Forest.
The aim of this study was to assess the impact of isolation on forest bird communities in agricultural land-  scapes in The Netherlands. We studied the avifauna of 235 small (0.1-39 ha) composed of mature deciduous trees in 1985. These were selected in the eastern and central/southern part of the country within 22 regions showing great differences in landscape structure, i.e., degree of isolation. Multiple regression analysis indicated that size was the best single predictor of species number and probability  of occurrence of most species. It turned out that the isolation variables, area of wood, number of woods, interpatch distance, and proximity and density of connecting elements, explained small but significant parts  of the residual variances in species number. No single species was significantly affected by the density of connecting elements. Biogeographical differences between two groups of regions were emphasized. Evidence of  four woodland- species suggested that regional abundance affected the probability of occurrence in small  isolates. 
Issues of grid%2Dcluster retrievals in swarm%2Dbased clustering.

Spectrum of Variable%2DRandom Trees.

Examining Dissimilarity Scaling in Ant Colony Approaches to Data Clustering.

Cocktail Ensemble for Regression.

To Select or To Weigh A Comparative Study of Linear Combination Schemes for SuperParent%2DOne%2DDependence Estimators.
 We conduct a large-scale comparative study on linearly combining superparent-one-dependence estimators (SPODEs), a popular family of semi-naive Bayesian classifiers. Altogether 16 model selection and weighing schemes, 58 benchmark data sets, as well as various statistical tests are employed. This paper’s main contributions are three-fold. First, it formally presents each scheme’s definition, rationale and time complexity; and hence can serve as a comprehensive reference for researchers interested in ensemble learning. Second, it offers bias-variance analysis for each scheme’s classification error performance. Third, it identifies effective schemes that meet various needs in practice. This leads to accurate and fast classification algorithms with immediate and significant impact on real-world applications. Another important feature of our study is using a variety of statistical tests to evaluate multiple learning methods across multiple data sets. 
Classifying under computational resource constraints anytime classification using probabilistic estimators.
In many online applications of machine learning, the computational resources available for classification will vary from time to time. Most techniques are designed to operate within the constraints of the minimum expected resources and fail to utilize further resources when they are available. We propose a novel anytime classification algorithm, anytime averaged probabilistic estimators (AAPE), which is capable of delivering strong prediction accuracy with little CPU time and utilizing additional CPU time to increase classification accuracy. The idea is to run an ordered sequence of very efficient Bayesian probabilistic estimators (single improvement steps) until classification time runs out. Theoretical studies and empirical validations reveal that by properly identifying, ordering, invoking and ensembling single improvement steps, AAPE is able to accomplish accurate classification whenever it is interrupted. It is also able to output class probability estimates beyond simple 0/1-loss classifications, as well as adeptly handle incremental learning.
Ehipasiko A Content%2Dbased Image Indexing and Retrieval System.

z%2DSVM An SVM for Improved Classification of Imbalanced Data.

To Select or To Weigh A Comparative Study of Model Selection and Model Weighing for SPODE Ensembles.
An ensemble of Super-parent-one-dependence Estimators (SPODEs) offers a powerful yet simple alternative to naive Bayes classifiers, achieving significantly higher classification accuracy at a moderate cost in classification efficiency. Currently there exist two families of methodologies that ensemble candidate SPODEs for classification. One is to select only helpful SPODEs and uniformly average their probability estimates, a type of model selection. Another is to assign a weight to each SPODE and linearly combine their probability estimates, a methodology named model weighing. This paper presents a theoretical and empirical study comparing model selection and model weighing for ensembling SPODEs. The focus is on maximizing the ensemble’s classification accuracy while minimizing its computational time. A number of representative selection and weighing schemes are studied, providing a comprehensive research on this topic and identifying effective schemes that provide alternative trades-off between speed and expected error.  
Variable Randomness in Decision Tree Ensembles.

Ensemble Selection for SuperParent%2DOne%2DDependence Estimators.

Maximizing Tree Diversity by Building Complete%2DRandom Decision Trees.

On the Application of ROC Analysis to Predict Classification Performance Under Varying Class Distributions.
We counsel caution in the application of ROC analysis for prediction of classifier performance under varying class distributions. We argue that it is not reasonable to expect ROC analysis to provide accurate prediction of model performance under varying distributions if the classes contain causally relevant subclasses whose frequencies may vary at di#erent rates or if there are attributes upon which the classes are causally dependent.
Improving the Centered CUSUMS Statistic for Structural Break Detection in Time Series.

Matching Model Versus Single Model A Study of the Requirement to Match Class Distribution Using Decision Trees.

Model Stability A key factor in determining whether an algorithm produces an optimal model from a matching distribution.

A Study of AdaBoost with Naive Bayesian Classifiers Weakness and Improvement.

A Study on the Effect of Class Distribution Using Cost%2DSensitive Learning.

Issues in Classifier Evaluation using Optimal Cost Curves.

An Instance%2DWeighting Method to Induce Cost%2DSensitive Trees.
Abstract — Class imbalance is one of the challenging problems for machine learning algorithms. When learning from highly imbalanced data, most classifiers are overwhelmed by the majority class examples, so the false negative rate is always high. Although researchers have introduced many methods to deal with this problem, including resampling techniques and costsensitive learning (CSL), most of them focus on either of these techniques. This study presents two empirical methods that deal with class imbalance using both resampling and CSL. The first method combines and compares several sampling techniques with CSL using support vector machines (SVM). The second method proposes using CSL by optimizing the cost ratio (cost matrix) locally. Our experimental results on 18 imbalanced datasets from the UCI repository show that the first method can reduce the misclassification costs, and the second method can improve the classifier performance. I.
An Empirical Study of MetaCost Using Boosting Algorithms.
MetaCost is a recently proposed procedure that converts an error-based learning algorithm into a cost-sensitive algorithm. This paper investigates two important issues centered on the procedure which were ignored in the paper proposing MetaCost. First, no comparison was made between MetaCost&#039;s final model and the internal cost-sensitive classifier on which MetaCost depends. It is credible that the internal cost-sensitive classifier may outperform the final model without the additional computation required to derive the final model. Second, MetaCost assumes its internal cost-sensitive classifier is obtained by applying a minimum expected cost criterion. It is unclear whether violation of the assumption has an impact on MetaCost&#039;s performance. We study these issues using two boosting procedures, and compare with the performance of the original form of MetaCost which employs bagging.
A Comparative Study of Cost%2DSensitive Boosting Algorithms.
This paper describes a study of different adaptations of boosting algorithms for cost-sensitive classification. The purpose of the study is to improve our understanding of the behavior of various cost-sensitive boosting algorithms and how variations in the boosting procedure affect misclassification cost and high cost error. We find that boosting can be simplified for cost-sensitive classification. A new variant, which excludes a factor used in ordinary boosting, performs best at minimizing high cost errors and it almost always performs better than AdaBoost. We also find that cost-sensitive boosting seeks to minimize high cost errors rather than cost, and a minimum expected cost criterion, applied during classification, greatly enhances the performance of all cost-sensitive adaptations of boosting algorithms. We show a strong correlation between an algorithm that produces small model size and its success in reducing high cost errors. For a recently proposed method, AdaCost,...
Lazy Bayesian Rules A Lazy Semi%2DNaive Bayesian Learning Technique Competitive to Boosting Decision Trees.
Lbr is a lazy semi-naive Bayesian classifier learning technique, designed to alleviate the attribute interdependence problem of naive Bayesian classification. To classify a test example, it creates a conjunctive rule that selects a most appropriate subset of training examples and induces a local naive Bayesian classifier using this subset. Lbr can significantly improve the performance of the naive Bayesian classifier. A bias and variance analysis of Lbr reveals that it significantly reduces the bias of naive Bayesian classification at a cost of a slight increase in variance. It is interesting to compare this lazy technique with boosting and bagging, two well-known state-of-the-art non-lazy learning techniques. Empirical comparison of Lbr with boosting decision trees on discrete valued data shows that Lbr has, on average, significantly lower variance and higher bias. As a result of the interaction of these effects, the average prediction error of Lbr over a range of learning tasks is at...
Improving the Performance of Boosting for Naive Bayesian Classification.
. This paper investigates boosting naive Bayesian classification. It first shows that boosting cannot improve the accuracy of the naive Bayesian classifier on average in a set of natural domains. By analyzing the reasons of boosting&#039;s failures, we propose to introduce tree structures into naive Bayesian classification to improve the performance of boosting when working with naive Bayesian classification. The experimental results show that although introducing tree structures into naive Bayesian classification increases the average error of naive Bayesian classification for individual models, boosting naive Bayesian classifiers with tree structures can achieve significantly lower average error than the naive Bayesian classifier, providing a method of successfully applying the boosting technique to naive Bayesian classification. 1 Introduction  For many KDD applications, the prediction (classification) accuracy is the primary concern. Recent studies on the boosting technique have brought...
Issues in Stacked Generalization.
Stacked generalization is a general method of using a high-level model to combine lower-level models to achieve greater predictive accuracy. In this paper we address two crucial issues which have been considered to be a `black art&#039; in classification tasks ever since the introduction of stacked generalization in 1992 by Wolpert: the type of generalizer that is suitable to derive the higher-level model, and the kind of attributes that should be used as its input. We find that best results are obtained when the higher-level model combines the confidence (and not just the predictions) of the lower-level ones.
Learning from Batched Data Model Combination Versus Data Combination.

Boosting Cost%2DSensitive Trees.
. This paper explores two techniques for boosting cost-sensitive trees. The two techniques differ in whether the misclassification cost information is utilized during training. We demonstrate that each of these techniques is good at different aspects of cost-sensitive classifications. We also show that both techniques provide a means to overcome the weaknesses of their base cost-sensitive tree induction algorithm. 1 Introduction  Cost-sensitive classification deals with situations where different types of incorrect prediction cost differently. The cost of incorrect predictions is more important than the number of incorrect predictions in many real world applications. For example, in medical diagnosis, the cost committed in diagnosing someone as healthy when one has a life-threatening disease is usually considered to be much higher than another type of error---of diagnosing someone as ill when one is in fact healthy. The former type of error is more serious than the latter type of error...
Boosting Trees for Cost%2DSensitive Classifications.
. This paper explores two boosting techniques for cost-sensitive tree classification in the situation where misclassification costs change very often. Ideally, one would like to have only one induction, and use the induced model for different misclassification costs. Thus, it demands robustness of the induced model against cost changes. Combining multiple trees gives robust predictions against this change. We demonstrate that ordinary boosting combined with the minimum expected cost criterion to select the prediction class is a good solution under this situation. We also introduce a variant of the ordinary boosting procedure which utilizes the cost information during training. We show that the proposed technique performs better than the ordinary boosting in terms of misclassification cost. However, this technique requires to induce a set of new trees every time the cost changes. Our empirical investigation also reveals some interesting behavior of boosting decision trees for cost-sensi...
Inducing Cost%2DSensitive Trees via Instance Weighting.

Model Combination in the Multiple%2DData%2DBatches Scenario.
The approach of combining models learned from multiple batches of data provide an alternative to the common practice of learning one model from all the available data (i.e., the data combination approach). This paper empirically examines the base-line behaviour of the model combination approach in this multiple-data-batches scenario. We find that model combination can lead to better performance even if the disjoint batches of data are drawn randomly from a larger sample, and relate the relative performance of the two approaches to the learning curve of the classifier used. The practical implication of our results is that one should consider using model combination rather than data combination, especially when multiple batches of data for the same task are readily available. Another interesting result is that we empirically show that the nearasymptotic performance of a single model, in some classification task, can be significantly improved by combining multiple models (derived from t...
Stacking Bagged and Dagged Models.
In this paper, we investigate the method of stacked generalization in combining models derived from different subsets of a training dataset by a single learning algorithm, as well as different algorithms. The simplest way to combine predictions from competing models is majority vote, and the effect of the sampling regime used to generate training subsets has already been studied in this context---when bootstrap samples are used the method is called bagging, and for disjoint samples we call it dagging. This paper extends these studies to stacked generalization, where a learning algorithm is employed to combine the models. This yields new methods dubbed bag-stacking and dag-stacking. We demonstrate that bag-stacking and dag-stacking can be effective for classification tasks even when the training samples cover just a small fraction of the full dataset. In contrast to earlier bagging results, we show that bagging and bag-stacking work for stable as well as unstable learning algorithms, a...
Stacked Generalizations When Does It Work%3F
Stacked generalization is a general method of using a high-level model to combine lower-level models to achieve greater predictive accuracy. In this paper we address two crucial issues which have been considered to be a `black art&#039; in classification tasks ever since the introduction of stacked generalization in 1992 by Wolpert: the type of generalizer that is suitable to derive the higher-level model, and the kind of attributes that should be used as its input. We demonstrate the effectiveness of stacked generalization for combining three different types of learning algorithms.
Discretisation in Lazy Learning Algorithms.

Decision Combination Based on the Characterisation of Predictive Accuracy.

The Characterisation of Predictive Accuracy and Decision Combination.
In this paper, we first explore an intrinsic problem that exists in the theories induced by learning algorithms. Regardless of the selected algorithm, search methodology and hypothesis representation by which the theory is induced, one would expect the theory to make better predictions in some regions of the description space than others. We term the fact that an induced theory will have some regions of relatively poor performance  the problem of locally low predictive accuracy.  Having characterised the problem of locally low predictive accuracy in Instance-Based and Naive Bayesian classifiers, we propose to counter this problem using a composite learner that incorporates both classifiers. The strategy is to select an estimated better performing classifier to do the final prediction  during classification. Empirical results show that the strategy is capable of partially overcoming the problem and at the same time improving the overall performance of its constituent classifiers. We pro...
Towards using a Single Uniform Metric in Instance%2DBased Learning.

An M%2Dof%2DN Rule Induction Algorithm and its Application to DNA Domain.

